Image display apparatus, control method of image display apparatus, and  program

ABSTRACT

An image display apparatus displays image data including a plurality of objects. The image display apparatus includes a display unit configured to focus-display an object from among the plurality of objects; an input unit configured to input annotation to the object; and a change unit configured to change a display of the annotation depending on the focus-displayed object if an object different from the object corresponding to the annotation that has been input by the input unit is focus-displayed.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image display apparatus, a controlmethod of an image display apparatus, and a program, and specificallyrelates to an image display apparatus and an image display method thatare suitable for browsing structured image data with other viewers bysharing a screen.

2. Description of the Related Art

There is a technique in which image data is displayed by using such as aprojector or an electronic blackboard, and data that is input byhandwriting in real time (referred to as “annotation data”, or simply“annotation”) is displayed superimposed on this displayed image data.Japanese Patent Laid-Open No. 2014-004783 discloses a technique ofincreasing a transmittance of the annotation with the increase of theelapsed time after the annotation is input if the annotation isdisplayed superimposed on the image data displayed on the screen such asthe electronic blackboard. Accordingly, old annotations gradually fadein accordance with the elapsed time, and if new annotations and the oldannotation are overlapped, a viewer of the screen can easily focus onthe new annotations.

However, image data that successively changes its display is included inthe image data. As an example of the image data that successivelychanges its display, there is a page in which “animation in presentationapplication” is set. A plurality of objects included in such a page arefocused in order according to a user's operation, and displayed on thescreen. Additionally, as another example of the image data thatsuccessively changes its display, there is also image data in which anobject specified by the user is focused on and displayed on the screen,among the plurality of objects included in the page.

In addition, the following methods serve as examples of thefocus-display: a method for displaying an object at a displaymagnification corresponding to a size of the object to which attentionis to be paid; a method for displaying the object to which attention isto be paid with highlights and displaying other objects in grayout; anda method for visibly displaying the object to which attention is to bepaid that has been hidden (has been invisible).

Here, it is contemplated that the annotation is entered for thesupplemental explanation of the object if an object among the pluralityof objects included in the page is being focus-displayed. Such kind ofannotation is desirably displayed during the focus-display of the objectthat is a target of the supplemental explanation. However, in theJapanese Patent Laid-Open No. 2014-004783, even such a kind ofannotation may gradually fade in accordance with the elapsed time.Additionally, if the annotation is entered while changing the area to bedisplayed with attention (the area to be focus-displayed), there arecases in which the annotations are displayed superimposed on the area towhich attention is to be paid, or the viewer will pay attention to anarea different from the area where a presenter wants the viewer to payattention.

SUMMARY OF THE INVENTION

The present invention provides an image display apparatus such that onlyan annotation that is necessary for an area to which attention is to bepaid is displayed if the annotation is entered while changing the areato be displayed is the focus of attention (focus-display).

According to an embodiment of the present invention, an image displayapparatus that displays image data including a plurality of objects isprovided that includes a display unit configured to focus-display anobject from among the plurality of objects; an input unit configured toinput annotation to the object; and a change unit configured to change adisplay of the annotation depending on the focus-displayed object if anobject different from the object corresponding to the annotation thathas been input by the input unit is focus-displayed.

According to the present invention, if the annotation is entered whilechanging the area to be displayed is the focus of attention (focusdisplay), the image display apparatus that displays only the annotationthat is necessary for the area to which attention is to be paid can beprovided. Hence, it is possible to prevent the annotations fromdisplayed while overlapping in the area to be paid attention, and toprevent the viewer's attention from turning to an area that is differentfrom the area to which a presenter wants the viewer to pay attention.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments (with reference to theattached drawings).

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic diagram in a case of performing a presentation byusing an image display apparatus.

FIG. 2 is a hardware block diagram of the image display apparatus.

FIG. 3 is a software block diagram of the image display apparatus.

FIG. 4 is a diagram illustrating a screen display example of a touch UIof the image display apparatus.

FIG. 5 is a diagram illustrating an example of a result for dividing anobject.

FIG. 6 is a table illustrating block information of each attribute andinput file information.

FIG. 7 is a diagram illustrating a sentence to be a target for sentencestructure tree generation processing.

FIG. 8 is a diagram illustrating the sentence structure tree that isgenerated in the processing of FIG. 7.

FIG. 9 is a flowchart illustrating the reproduction of application imagedata.

FIG. 10 is a flowchart illustrating mode switching processing.

FIGS. 11A to 11D are diagrams illustrating an example of annotationdisplay on the touch UI of the image display apparatus.

FIGS. 12E to 12H are diagrams illustrating an example of the annotationdisplay on the touch UI of the image display apparatus.

FIG. 13 is a flowchart illustrating processes of annotation generation.

FIG. 14 is a flowchart illustrating next selection processing.

FIGS. 15A to 15C are tables illustrating the display states of theobject.

FIGS. 16A and 16B are tables illustrating attribute information of theannotation.

FIG. 17 is a flowchart illustrating previous selection processing.

FIG. 18 is a flowchart illustrating change processing of the annotationduring the enlarged display.

FIGS. 19A and 19B are diagrams illustrating an example of the display ofthe annotation on the touch UI during the enlarged display.

FIG. 20 is a flowchart illustrating annotation change processing.

FIGS. 21A and 21B are tables illustrating the display order ofanimation.

FIGS. 22A and 22B are diagrams illustrating an example of the display ofthe annotation on the touch UI.

DESCRIPTION OF THE EMBODIMENTS

Hereinafter, a description will be given of embodiments for performingthe present invention with reference to attached drawings and the like.

First Embodiment

FIG. 1 is a conceptual diagram in a case of performing a presentation byusing an image display apparatus 100 according to the embodiment. In thepresent embodiment, a case in which the presentation is performed in aconference room in an office is supposed. The image display apparatus100 may be an information processing apparatus, for example, a portableinformation terminal such as a smart phone and a tablet personalcomputer. A presenter displays data in a predetermined format(hereinafter, referred to as “application image data”) by operating anapplication in the image display apparatus 100. An operation method ofthe application will be described below, and thus, the detaileddescription here will be omitted. The application image data displayedon the image display apparatus 100 is output to a projector as an RGB(red, green, and blue) signal. Specifically, the projector and the imagedisplay apparatus 100 are connected through an RGB cable, and the RGBsignal output from the image display apparatus 100 is input to theprojector through the RGB cable. The projector projects the input RGBsignal to a screen. In the present embodiment, application image datathat is identical to the application image data displayed on the imagedisplay apparatus 100 is projected. Therefore, the viewer can browse theapplication image data displayed on the image display apparatus 100 byviewing the screen, and sharing with a plurality of viewers.

Note that, in the present embodiment, although the viewer browses theapplication image data using the screen, the viewer may browse it usinga display included in the image display apparatus 100. Additionally, inthe present embodiment, although the image display apparatus 100 thatincludes a touch panel serving as an input unit is used, the presentinvention is not limited to the touch panel and other input units may beused if the operation of the image display apparatus 100 and the entryof annotation to the application image data are possible. Hereinafter,the entry of annotation will be simply referred to as the “annotation”.

FIG. 2 is a block diagram illustrating a hardware configuration of theimage display apparatus 100 according to the present embodiment. Theimage display apparatus 100 is configured by a main board 200, an LCD201, a touch panel 202, and a button device 203. Note that, in thepresent embodiment, the LCD 201 and the touch panel 202 collectivelyconfigure a touch UI 204. Components of the main board 200 include a CPU205, a wireless LAN module 206, a power source controller 207, and adisplay controller (DISPC) 208. Additionally, they include a panelcontroller (PANELC) 209, a ROM 210, a RAM 211, a secondary battery 212,a timer 213, and an RGB output controller 214. Note that the componentsare communicably connected to each other through buses (notillustrated).

The CPU (Central Processing Unit) 205 controls each device connected tothe bus, develops a software module 300 stored in the ROM (Read OnlyMemory) 210 into the RAM 211 and executes it. Note that a softwaremodule 300 will be described below. The RAM (Random Access Memory) 211functions as a main memory of the CPU 205, a working area, a video imagearea to be displayed on the LCD 201, and a storage area of theapplication image data.

The display controller (DISPC) 208 switches the video image output thatwas developed into the RAM 211 at high speed in response to a request ofthe CPU 205, and outputs a synchronous signal to the LCD 201.Consequently, the video image of the RAM 211 is output to the LCD 201 insynchronization with the synchronous signal of the DISPC 208, and thevideo image is displayed on the LCD 201.

The panel controller (PANELC) 209 controls the touch panel 202 and thebutton device 203 in response to the request of the CPU 205. Hence, adepressed position on the touch panel 202 by an indicator, for example,a finger or a stylus pen, or a key code that was depressed on the buttondevice 203 is reported to the CPU 205. Depressed position informationconsists of a coordinate value indicating an absolute position of thetouch panel 202 in the horizontal direction (hereinafter, referred to asthe “x-coordinate”) and a coordinate value indicating the absoluteposition of the touch panel 202 in the vertical direction (hereinafter,referred to as the “y-coordinate”). Note that the touch panel 202 iscapable of detecting the depression at a plurality of points, and inthat case, the depression position information by the total number ofthe depressions is reported to the CPU 205.

The power source controller 207 is connected to an external power source(not illustrated) and supplied with electric power. Thus, the electricpower is supplied to the entire image display apparatus 100 whilecharging the secondary battery 212 connected to the power sourcecontroller 207. If the electric power is not supplied from the externalpower source, the electric power from the secondary battery 212 issupplied to the entire image display apparatus 100.

The wireless LAN module 206 establishes wireless communication with awireless LAN module on a wireless access point (not illustrated)connected to a LAN (not illustrated) that is built into an office (e.g.,facilities) based on the control of the CPU 205, and mediatescommunication with the image display apparatus 100. The wireless LANmodule 206 may be, for example, IEEE802.11b.

The timer 213 generates timer interruptions to a gesture eventgeneration unit 301 described below, based on the control of the CPU205. Additionally, the image display apparatus 100 includes ageomagnetic sensor (not illustrated) and an acceleration sensor (notillustrated), which are respectively connected to the buses. The timer213 detects the tilt of the image display apparatus 100 based on thecontrol of the CPU 205, and if the image display apparatus 100 obtainsthe tilt equal to or greater than a predetermined tilt, the timer 213changes the orientation of the image display apparatus 100, andtransmits an instruction to draw on the LCD 201 to a drawing unit 303described below. If the orientation of the image display apparatus 100is changed, the CPU 205 replaces the width and height of the LCD 201 andperforms the following processing.

Specifically, the RGB output controller 214 switches the video imageoutput that was deployed in the RAM 211 at high speed, and transmits theRGB video image signal to an external display device, for example, theprojector. Consequently, the video image of the RAM 211 is output to theexternal display device, for example, the projector, and an image thatis identical to the image on the LCD 201 is displayed on the screenprojected by the projector.

Next, a description will be given of a software module related to theoperation control of the application image data of the image displayapparatus 100 according to the present embodiment, with reference toFIG. 3 and FIG. 4. FIG. 3 is a block diagram illustrating aconfiguration of a software module 300 that is executed in the CPU 205of the image display apparatus 100. Additionally, FIG. 4 is a diagramillustrating an example of a screen display on the touch UI 204 of theimage display apparatus 100 according to the present embodiment.

First, each module that configures the software module 300 will bedescribed. The gesture event generation unit 301 receives a touch inputof the user, generates a variety of gesture events, and transmits thegenerated gesture events to a gesture event processing unit 302. Notethat, in the present embodiment, the various gesture events include atouch depression event, a touch release event, a single tap event, adouble tap event, a swipe event, a pinch-in event, and a pinch-outevent. Here, the various gesture events will be described.

In the touch depression event, the coordinate values of the latest touchcoordinates and the touch coordinate number are transmitted to thegesture event processing unit 302. Note that the touch coordinatesindicate the coordinates of one point where the user's finger istouching the touch panel 202, and it has a set of coordinate valuesrepresented by the x-coordinate and the y-coordinate. Additionally, thetouch coordinate number indicates the number of the touch coordinates atwhich the user's finger touched the touch panel 202. The touchcoordinates are updated if the user's finger touches the touch panel202, the finger is moved, the finger is detached from the panel, or aninterruption from the timer 213 is generated.

In the touch release event, the coordinate values and the coordinatenumber of the latest touch coordinates at the detachment timing of theuser's finger from the touch panel 202 are transmitted to the gestureevent processing unit 302. In the single tap event, the coordinatevalues of the latest touch coordinates are transmitted to the gestureevent processing unit 302. The single tap indicates that the touchrelease event has been generated within a predetermined period of timeafter the aforementioned touch depression event. In the double tapevent, the coordinate values of the latest touch coordinates aretransmitted to the gesture event processing unit 302. The double tapindicates that the aforementioned single tap event has been generatedtwice within a predetermined period of time.

In the swipe event, the coordinate values of the latest touchcoordinates, and the movement distance calculated based on thedifference between the latest and the previous coordinates aretransmitted. The swipe is a moving (so as to slide) operation of afingertip in one direction while keeping the fingertip in contact withthe touch panel 202. In the pinch-in event, a reduction ratio of thepinch-in calculated based on center coordinate values between two pointsof latest touch coordinates and a reduction distance of a straight lineconnecting the two points of the touch coordinates is transmitted. Thepinch-in is an operation in which two finger tips approach (so as topinch) each other while keeping the two finger tips in contact with thetouch panel 202. In the pinch-out event, an enlarged ratio of thepinch-out calculated based on center coordinate values between twopoints of the latest touch coordinates and an enlarged distance of thestraight line connecting the two points of the touch coordinates istransmitted. The pinch-out is an operation in which two finger tipsseparate (so as to spread the fingers) from each other while keeping thetwo finger tips in contact with the touch panel 202. Note that themechanism of the generation of the above-mentioned gesture events is aknown technique, and thus further detailed description will be omitted.

The gesture event processing unit 302 receives the gesture events thatwere generated in the gesture event generation unit 301, and executesthe operation control according to each gesture event and a sentencestructure described in the application image data. The drawing unit 303draws the application image data to the LCD 201 in response to theexecution result for the gesture event processing unit 302. A displaymethod for the application image data will be described below.

If a display change event processing unit 304 receives the single tapevent, it determines whether or not the coordinate values of the touchcoordinate of the single tap event are positioned on any one of a modeswitching button 401, a next button 402, a previous button 403, adrawing button 404 shown in FIG. 4. Subsequently, if the coordinatevalues of the touch coordinates of the single tap event are positionedon the “mode switching button” 401, mode switching processing describedbelow is performed. Additionally, if the coordinate values of the touchcoordinates are positioned on the “next button” 402, “the next selectionprocessing” (next button selection processing) described below isperformed, and if the coordinate values of the touch coordinates arepositioned on the “previous button” 403, “previous selection processing”(previous button selection processing) described below is performed. Ifthe coordinate values of the touch coordinates are positioned on the“drawing button” 404, “annotation drawing processing” (drawing buttonselection processing) described below is performed. However, the“drawing button” 404 is displayed only during a presentation displaymode described below. The “next selection processing” and the “previousselection processing” are performed in a display order control unit 305and an attention area control unit 306 in the display change eventprocessing unit 304. Specifically, according to the order of the objectsspecified by the display order control unit 305, the objects arefocus-displayed in order in the forward direction in the “next selectionprocessing” or the objects are focus-displayed in order in the backwarddirection in the “previous selection processing”. The “annotationdrawing processing” is performed in an annotation display control unit307.

A swipe event processing unit 308 performs processing on the swipeevent. If the gesture event processing unit 302 receives the swipeevent, the swipe event processing unit 308 moves the start point of thepage 400 at the coordinates on the touch UI 204 in accordance with themovement distance of the swipe event. Subsequently, it updates thedisplay state of the touch UI 204 based on this. Anenlargement/reduction event processing unit 309 performs processing onthe pinch-in event or the pinch-out event. The gesture event processingunit 302 receives the pinch-in event or the pinch-out event, controlsthe start point and a display magnification of the page 400 inaccordance with the reduction ratio or the enlargement ratio of theaforementioned two events, and updates the display state of the touch UI204.

Next, a description will be given of a generation method of theapplication image data, which is data in a predetermined format for thedisplay on the image display apparatus 100. The application image datais obtained by an image reading unit in a MFP (not illustrated) that isa multifunction printer realizing a plurality of types of functions(e.g., a copy function, a printing function, and a transmittingfunction). Alternatively, the application image data is generated byrendering a document generated by application software on a client'spersonal computer (not illustrated), which is an information processingapparatus including a personal computer, inside the MFP. The MFP and theclient's personal computer are connected to a LAN (not illustrated) thatis built in the office (e.g., facilities), and they can transmit andreceive the data with each other.

First, object division processing, which divides the bitmap image datathat was obtained at the image reading unit of the MFP or generated bythe application software on the client's personal computer into theobjects for each attribute, is performed. The type of the attribute ofthe object after dividing the object indicates characters, photographs,and graphics (drawings, line drawings, tables, and lines). The type ofthe object (characters, photographs, and graphics) is determined withrespect to each of the divided objects.

Next, it is determined whether or not the object is characters, and ifthe object is characters, OCR processing is further performed, andcharacter coded data (character coded data as a result for the OCR) isobtained. Note that the OCR is a known technique and the detaileddescription will be omitted. With respect to each of the dividedobjects, an area of the object is cut out from the bit map image data byusing positional information of the object, and the object image isgenerated. Resolution of the object is converted according to the typeof the attribute of the object so as to maintain a suitable qualitywhile suppressing an amount of the data.

Next, the resolution of the bitmap image data is converted, and abackground image having a resolution lower than that of the bitmap imagedata is generated. In the present embodiment, the resolution isone-fourth of the bit map image data by using a nearest neighbor method,that is, the background image having 150 dpi is generated if the bit mapimage data has 600 dpi. Note that the method of converting resolution isnot limited to the nearest neighbor method, and, for example, a highlyaccurate interpolation method including a bilinear method or a bicubicmethod may be used. Subsequently, a JPEG-compressed background image byusing a background image having a resolution lower than the generatedbitmap image data is generated. Each of the object data, the backgroundimage data, and the character coded data is obtained based on a sentencestructure tree described below, and application image data that can bedisplayed on the image display apparatus 100 is generated. Note that themethod for generating application image data is a known technique, andthus, more detailed description will be omitted.

Next, the object division will be described in detail with reference toFIG. 5 and FIG. 6. FIG. 5 is a diagram illustrating an example of aresult for dividing the bitmap image data into a plurality of objects byobject division processing. FIG. 6 is a table illustrating blockinformation and input file information for each object if the object isdivided.

First, the object division processing is performed on the input image(the left in FIG. 5), and the objects are divided into rectangularblocks for each attribute (the right in FIG. 5). As described above,characters, photographs, graphics (drawings, line drawings, tables, andlines), and the like are the attribute of the rectangular block. As oneapproach to the object division processing, for example, there is amethod in the following.

First, the image data stored in the RAM (not illustrated) in the MFP isbinarized into black and white, and a pixel block surrounded by blackpixel outlines are extracted. Subsequently, the size of the extractedblack pixel block is evaluated, and the outlines tracing a white pixelblock inside the black pixel block having a size of a predeterminedvalue or more is performed. As long as the pixel block in the inside hasa size of the predetermined value or more as a result for evaluating thesize of the white pixel block and performing the outline tracing on theblack pixel block inside the white pixel block, the extraction of thepixel block in the inside and the outline tracing are recursivelyperformed. The size of the pixel block is evaluated by, for example, thearea size of the pixel block. The rectangular block that circumscribesthe pixel block obtained in this way is generated, and the attribute isdetermined based on the size and shape of the rectangular block. Forexample, the rectangular block in which the aspect ratio is close to 1and the size is in a fixed range serves as a character correspondingblock may be a character area rectangular block. Additionally, if thecharacter corresponding blocks adjacent each other are regularlyaligned, a new rectangular block that summarizes this charactercorresponding blocks is generated, and the new rectangular block servesas a character area rectangular block. Additionally, the black pixelblock that includes flat pixel blocks, or the white pixel blocks havinga fixed size or more and square shape in the aligned state serves as agraphic area rectangular block, and other amorphous pixel blocks serveas photograph area rectangular blocks.

As described above, regarding each of the rectangular blocks generatedas described above, a table illustrating the block information of theattribute and the like and input file information shown in FIG. 6 isgenerated. In FIG. 6, the attribute, the coordinate-X and thecoordinate-Y of the position, the width W, the height H, and the OCRinformation of each block are included in the block information. Thevalues of 1 to 3 are given to attributes. In the present embodiment, theattribute 1 indicates the character area rectangular block, attribute 2indicates the photograph area rectangular block, and the attribute 3indicate the graphic area rectangular block. The coordinate-X and thecoordinate-Y are the X and Y coordinates at the start points of eachrectangular block in the input image (coordinates at the upper leftcorner). The width W is the width of the rectangular block and in theX-coordinate direction and the height H is the height of the rectangularblock in the Y-coordinate direction. The OCR information indicates thepresence or absence of pointer information to the character encoded databy the OCR processing. Moreover, the total number of the blocks Nindicating the number of the rectangular blocks is also stored as theinput file information.

The block information for each rectangular block is used in theapplication image data generation processing described below.Additionally, a relative positional relationship if overlapping aspecific area and other areas can be specified by the block information,and overlapping each area without impairing the layout of the inputimage is made possible. Note that the method of dividing the object is aknown technique, and thus more detailed description will be omitted.

Next, with reference to FIG. 7 and FIG. 8, a description will be givenof sentence structure tree generation processing. FIG. 7 is a diagramillustrating one example of a sentence to be the sentence structuretree, and FIG. 8 is a diagram illustrating the sentence structure treegenerated by the sentence structure tree generation processing. Notethat the sentence structure tree generation is processing that isperformed in a process of the application image data generation by theMFP.

First, the MFP regroups the rectangular blocks in unit of therectangular block on the basis of relevance in the vertical direction.For example, in the sentence shown in FIG. 7, rectangular blocks T1 andT2 are arranged in parallel in the horizontal direction at the top. Ahorizontal direction separator S1 exists under the rectangular blocks T1and T2, and rectangular blocks T3, T4, T5, T6, and T7 exist under thehorizontal direction separator S1. The rectangular blocks T3, T4, and T5are arranged in the vertical direction from top to bottom in the lefthalf portion in the lower area of the horizontal direction separator S1,and the rectangular blocks T6 and T7 are arranged from top to bottom inthe right half portion in the lower area of the horizontal directionseparator S1. If the processing of a grouping based on the relevance inthe vertical direction is executed, the rectangular blocks T3, T4, andT5 are summarized in one group (rectangular block) V1, and therectangular blocks T6 and T7 are summarized in one group (rectangularblock) V2. The groups V1 and V2 are on the same layer.

Next, the presence or absence of a separator in the vertical directionis checked. The separator is an object having a line attribute and has afunction of explicitly dividing the block. If the separator is detected,the area of the input image is divided into left and right areas alongthe separator as the boundary, in the layer to be processed. Note that,in the present embodiment, the separator in the vertical direction doesnot exist in FIG. 7.

Next, it is determined whether or not the sum of the group height in thevertical direction becomes equal to the height of the input image. Thatis, in a case where the grouping in the horizontal direction isperformed while moving the area to be processed in the verticaldirection (for example, from top to bottom), if the processing of theentire input image has ended, the finish determination of the processingmay be carried out by using a fact in which the sum of the group heightbecomes the height of the input image. If the grouping has ended, theprocess ends as it is, and if the grouping has not ended, the processproceeds to the grouping processing based on the relevance in thehorizontal direction.

By the grouping processing in the horizontal direction, for example, therectangular blocks T1 and T2 shown in FIG. 7 are summarized into onegroup (rectangular block) H1, and the rectangular blocks V1 and V2 aresummarized into one group (rectangular block) H2. Note that the groupsH1 and H2 are on the same layer. Next, the presence or absence of theseparator in the horizontal direction is checked. If the separator isdetected, the area of the input image is divided into top and bottomareas along the separator, which serves as the boundary, in the layer tobe processed. In FIG. 7, the separator S1 of the horizontal direction ispresent. The results for the process described above are registered as atree shown in FIG. 8.

In FIG. 8, the bit map image data V0 of one page that was input has thegroups H1 and H2 and the separator S1 on the highest layer, and therectangular blocks T1 and T2 on the second layer belong to the group H1.The groups V1 and V2 on the second layer belong to the group H2, therectangular blocks T3, T4, and T5 on the third layer belong to the groupV1, and rectangular blocks T6 and T7 on the third layer belong to thegroup V2. In the present embodiment, V0 represents the page, and groupspresent in the lower layers of V0 become the objects.

Finally, it is determined whether or not the sum of the group length inthe horizontal direction is equal to the width of the input image.Accordingly, the determination of completion with respect to thegrouping in the horizontal direction is performed. If the group lengthin the horizontal direction is the page width, the sentence structuretree generation processing ends. If the group length in the horizontaldirection is not the page width, the process returns to the initialprocess, and again, on one upper layer, the process is repeated from therelevancy checking of the vertical direction.

Next, with reference to FIG. 4 and FIG. 9, a description will be givenof processing at the time of reproducing the application image data bythe image display apparatus 100 according to the present embodiment.FIG. 9 is a flowchart illustrating the reproduction of the applicationimage data by the image display apparatus 100.

First, in step S901, the image display apparatus 100 receives theapplication image data from the MFP via the wireless LAN module 206, andstores the received application image data in the RAM 211. Next, in stepS902, the syntax of the application image data stored in the RAM 211 isanalyzed, and the head page is read. Next, in step 5903, the drawingunit 303 performs rendering of the background included in the head pagethat has been read according to the coordinates of the start point, thewidth and the height in the area information, and updates the displaystate of the touch UI 204.

At this time, as shown in page 400 of FIG. 4, the display magnificationof the head page is controlled such that the height of the page 400matches that of the touch UI 204 or the width of the page 400 matchesthat of the touch UI 204. Additionally, if the width or the height ofthe page upon the reduction of the page at the display magnification issmaller than the touch UI 204, the start point of the page 400 iscontrolled in the coordinates on the touch UI 204 such that the page isdisplayed in the center of the touch UI 204. Thus, the display controlmethod for displaying the entire page in the touch UI 204 is referred toas a “page display mode”, in the present embodiment.

Next, with reference to FIG. 10, FIGS. 11A to 11D, FIGS. 12E to 12H, andFIGS. 15A to 15C, a description will be given of mode switchingprocessing executed in the display change event processing unit 304.FIG. 10 is a flowchart of the mode switching processing. FIGS. 11A to11D and FIG. 12E to 12H are examples of the screen display of the touchUI 204 in the image display apparatus 100 according to the presentembodiment. FIGS. 15A to 15C are tables illustrating the display statesof the objects of the examples of the screen display on the touch UI 204in FIGS. 11A to 11D and FIGS. 12E to 12H.

First, in step S1001, the display change event processing unit 304obtains the display mode being set in the image display apparatus 100 atthe point in time if that instructions for the mode switching buttonhave been provided. The display mode indicates a method in which theimage display apparatus 100 displays the application image data on thetouch UI 204, and the image display apparatus 100 according to thepresent embodiment has the following two display modes. A first displaymode is the page display mode suitable for displaying the entire page.As shown in FIGS. 11A to 11D and FIGS. 12E to 12H, a second display modeis a presentation display mode that is suitable for displaying a part ofthe areas in the page (that is, each object in the page image) withhighlights so as to facilitate focusing user's attention (focusdisplay). As described above, immediately after the image displayapparatus 100 has received the application image data, the page displaymode is set. The presentation display mode is a mode that displays thearea of attention on the page 400 such that each object in the page 400is displayed with highlights, as shown in FIGS. 11A to 11D and FIGS. 12Eto 12H. Note that, in the present embodiment, there are cases in whichthe area of attention is referred to as the attention target, or theattention object, but these have the same meaning.

Here, a screen that is displayed if the character object 1101 isselected as the object in the area of attention is shown in FIG. 11B.Note that the broken lines surrounding the objects 1101 to 1104 in FIG.11A are drawn in order to simplify the description, and they are notpresent on the actual page 400. Additionally, in the present embodiment,in order to clearly distinguish the display states of the object, atable that assigns IDs to the objects is generated. For example, theobjects 1101 to 1104 in FIGS. 11A to 11D and FIGS. 12E to 12H arerespectively assigned to the object IDs 1 to 4, and the display statesof the corresponding object IDs are shown in tables of FIGS. 15A to 15Cdescribed below. Note that the object IDs of FIGS. 15A to 15C aregenerated in the display order, based on the sentence structure tree.That is, if the objects 1101 to 1104 are displayed in order by thesentence structure tree (not illustrated) of the application image datain FIGS. 11A to 11D and FIGS. 12E to 12H, the object IDs 01 to 04 areassigned associated with the order of the objects 1101 to 1104.

In the present embodiment, as shown in FIG. 11B, with respect to theobjects 1102 to 1104 other than the object 1101 of the attention target,masks 1105 to 1107 that are respectively opaque are displayedsuperimposed on the page 400. Accordingly, only the object 1101 of thetarget of attention is made to be easily seen. Because the objects otherthan the object of the attention target are not displayed by displayingsuch masks superimposed, the object of the target of attention ishighlighted, so that the user can easily specify the area of the objectthat is a display target. Note that, in the present embodiment, althoughwhite opaque masks are displayed superimposed on the objects other thanthe object of the target of attention, it may be possible to display amask having a color that matches the background by obtaining thebackground color in advance if the background has a color other thanwhite.

First, in step S1001, the display change event processing unit 304determines the display mode being set (current) if an instruction isprovided to the mode switch button 401 is either of the presentationdisplay mode or the page display mode. Subsequently, if the display modebeing set is the presentation display mode, the process proceeds to stepS1002, and if it is the page display mode, the process proceeds to stepS1005.

Next, in step S1002, the display change event processing unit 304 setsthe mask 1105 in FIG. 11B to the non-display (mask OFF), and switchesthe mode to the page display mode. Next, in step S1003, the displaychange event processing unit 304 controls the display magnification ofpage 400 in accordance with the height of the touch UI 204, controls thestarting point of the page 400, and determines the display range of thepage. Subsequently, in step S1004, the display change event processingunit 304 updates the display state of the touch UI 204 based on thedisplay range of the determined page. In contrast, if the display modebeing set is the page display mode, the display change event processingunit 304 switches the display mode to the presentation display mode andchanges to the setting that displays the masks 1105 to 1107 (mask ON),in step S1005.

Next, in step S1006, the display change event processing unit 304 readsthe head object on the page 400, and obtains the start point, the width,and the height of the head object. Here, the head object is the objectthat was first read, in the sentence structure tree of the applicationimage data. Next, in step S1007, the attention area controlling unit 306in the display change event processing unit 304 performs attention areadetermination processing, and updates the display state of the touch UI204 based on the mask area determined in step S1004.

Note that, in the attention area determination processing of step S1007,the head object that has been read in step S1006 serves as the area ofattention, and the object in which the masks are displayed superimposedon the touch UI 204 is determined. For example, in FIG. 11B, because themasks 1105 to 1107 are respectively covered on the objects 1102 to 1104other than the object 1101 that is the display target at this time, theuser can easily identify the object 1101 that is the target ofattention. Additionally, a table showing the display state of the objectat this time is generated as shown in FIG. 15A. Specifically, an“attention state flag” of the object 1101 (object ID: 01) is set to 1,which indicates a state of focusing attention, and an “already displayedflag” thereof is set to 1, which indicates a state in which the objecthas already been read and displayed. Additionally, the attention stateflags of the objects 1102 to 1104 (object IDs: 02 to 04) are set to 0,which indicates a state of not focusing attention, and the alreadydisplayed flags thereof are set to 0, which indicates a state in whichthe object has not yet been read and displayed. Accordingly, theapplication is able to clearly distinguish the object area that isdisplayed with attention, the object area that is not displayed withattention by using the opaque mask, and an object area that is coveredwith a semi-opaque mask described below.

Next, with reference to FIGS. 11A to 11D, FIGS. 12E to 12H, FIG. 13, andFIGS. 16A and 16B, a description will be given of annotation displayprocessing performed in the annotation display control unit 307. FIG. 13is a flowchart of the annotation generation. FIGS. 16A and 16B aretables illustrating the attribute information of the annotation in theexample of the annotation display in FIGS. 11A to 11D, and FIGS. 12E to12H.

First, in step S1201, if the drawing button 404 is tapped during thepresentation display mode, except for the drawing button 404 the displaychange event processing unit 304 turns OFF the input to the buttons 401to 403, and switches the mode to an annotation entry mode. Note that, inthe present embodiment, the drawing button 404 is displayed only duringthe presentation display mode and the annotation entry mode, and it isimpossible to tap it during the page display mode. The annotation entrymode is, for example, a mode in which entry is manually performed in thepage 400 shown in FIG. 11B via the touch UI 204.

Here, FIG. 11C illustrates an example of the screen that is displayed ifthe user enters the annotation in the page 400. A rectangular areasurrounded by broken lines in FIG. 11C is an area of the enteredannotation 1108. However, the broken lines in FIG. 11C are drawn inorder to clearly show the annotation area, and they are not present onthe actual page 400. Additionally, in the present embodiment, a table inFIG. 16A described below is generated for controlling the display stateof the annotation.

Next, in step S1202, it is determined whether or not the drawing button404 has been tapped. If the drawing button 404 has been tapped (YES),the process proceeds to step S1203, and if the drawing button 404 hasnot been tapped (NO), the process proceeds to step S1207. Note that, inthe present embodiment, if the drawing button 404 has been tapped duringthe annotation entry mode, the process shifts to the finish processingof the annotation entry mode. That is, as long as the drawing button 404has not been tapped again, the process does not shift to thepresentation display mode from the annotation entry mode.

Next, in step S1203, the annotation image is generated from the enteredannotation drawing. In the present embodiment, a png file in which thetransmittance is freely settable is generated as the annotation image.However, the present invention is not limited to the png file, andanother file may be used if it is an image file in which thetransmittance is settable. Next, in step S1204, the attributeinformation of the annotation image generated in step S1203 isgenerated. For example, the attribute information of the annotation 1108in FIG. 11C is generated as a table shown in FIG. 16A.

Here, a description will be given of the table illustrating theattribute information of the annotation in FIG. 16A. Regarding the“annotation ID” in FIG. 16A, a unique value is assigned to eachannotation image to be generated. The “page” indicates a page of theapplication image data on which the annotation image is entered. The“corresponding object ID” is an object ID in which the attention stateflag in FIGS. 15A to 15C is 1 if the annotation image is entered. Thatis, it indicates the ID of the object displayed with attention if theannotation image is entered. The “transmittance” indicates thetransmittance of the entered annotation image, and if the transmittanceis 0[%], it indicates an opaque state that is completely visible, and ifthe transmittance is 100[%], it indicates a transparent state that iscompletely invisible. From the above, the annotation 1108 in FIG. 11C isan annotation image that was entered if the object 1101 (object ID: 01)is displayed for attention, and thus the corresponding object ID of theattribute information of the annotation becomes 01. Additionally, inFIG. 11C, because the object 1101 is the target of attention, thetransmission attribute of the annotation 1108 becomes the opaque state(0%). Note that the detailed description of the control of thetransmittance will be described below.

The annotation image and the attribute information of the annotation areheld as a set of the annotation. Subsequently, the attribute informationof the annotation is reentered, and thus the control of the annotationimage can be performed. Additionally, the attribute information of theannotation also holds information about an entry position of theannotation. Specifically, the attribute information about the annotationincludes the coordinate-X, the coordinate-Y, the width W, and the heightH of the position of each annotation, in a state similar to the blockinformation of the object. In FIGS. 16A and 16B, the positionalinformation of the annotation will be omitted for simplicity.Hereinafter, if the annotation is disclosed, it will be considered asdisclosing the set of the annotation image and the attribute informationof the annotation.

Next, in step S1205, the image display apparatus 100 finishes theannotation entry mode, and switches the mode to the presentation displaymode. In the present embodiment, the process shifts to the display stateof the object that is immediately before the shift to the annotationentry mode in step S1201. For example, if the state immediately beforethe shift to the annotation entry mode is a state in which the object1101 in FIG. 11B is displayed with attention, the process shifts to thepresentation display mode that is the same display state of the objectthereof. Next, in step S1206, the display state of the touch UI 204 isupdated based on the generated annotation. For example, if theannotation 1108 is entered in FIG. 11B, the touch UI 204 changes to thedisplay state in FIG. 11C.

In contrast, in step S1207, the annotation display control unit 307determines whether or not a touch depression event (input) on the touchUI 204 has been detected. If the touch depression event has beendetected (YES), the process proceeds to step S1208, and if the touchdepression event has not been detected (NO), the process returns to stepS1202.

Subsequently, in step S1208, the drawing processing of the annotation isperformed based on the coordinate values by the touch input to the touchUI 204 in step S1207. For example, in response to the touch input, thedrawing display of the annotation 1108 shown in FIG. 11C is performed.Note that, in the present embodiment, although a line width and a colorof the annotation are steadily displayed, those values can be freelychanged by setting change by the user (not illustrated).

Next, with reference to FIGS. 11A to 11D, FIGS. 12E to 12H, FIG. 14,FIGS. 15A to 15C, and FIGS. 16A and 16B, a description will be given of“next selection processing (next button selection processing)” that isexecuted if the “next button” 402 is tapped (specified) by the user.FIG. 14 is a flow chart of the next selection processing.

First, the display change event processing unit 304 obtains the displaymode that is set in the image display apparatus 100 if the “next button”402 is tapped. Subsequently, in step S1301, it is determined that theobtained display mode is either of the presentation display mode or thepage display mode. If the obtained display mode is the presentationdisplay mode, the process proceeds to step S1302, and if it is the pagedisplay mode, the process proceeds to step S1308. Next, in step S1302,the display order control unit 305 selects the object to be displayednext based on the sentence structure tree from among all the objects inthe page currently read, and reads the object.

Here, the display order of the objects will be described. In the presentembodiment, the display order on the document tree structure is theobject belonging to the lowest layer of the upper layer of the heading,subsequently the object belonging to the same layer of the object.Moreover, after finishing the display of all the objects belonging tothe same layer, the display is in the order of the object belonging tothe lowest layer of the next upper layer, and subsequently the objectbelonging to the same layer.

For example, in FIG. 8, V0 represents the page, and the object to beread first is T1. If the “next selection processing” is performed in astate where the object of T1 is displayed on the touch UI 204 in thepresentation display mode, the object of the T2 is read because the T1has the same layer. Moreover, if the “next selection processing” isperformed in a state where the object of T2 is displayed, S1 that ispositioned on the next higher layer is read because the next object isnot present in the layer same as T2. In this case, S1 has no lowerlayer, and therefore S1 is positioned on the lowest layer. Note that, inthe present embodiment, it is also possible to select only the objecthaving a specific attribute including the object of the characterattribute, or possible to select the object other than only specificattributes. Additionally, the order of the object IDs shown in FIGS. 15Ato 15C is generated corresponding to the display order.

Next, in step S1303, the display order control unit 305 determineswhether or not the next object can be read in step S1302. That is, it isdetermined whether or not the object is finished depending on thepresence or absence of a selectable object. If the next object can beread (if the selectable objects was present: NO), the object that hasbeen read serves as an object to be processed, and the process proceedsto step S1304. In contrast, if the next object cannot be read (if theselectable object was absent: YES), the process proceeds to step S1307.

Next, in the attention area determination processing in step S1304, thenext object that was read as the object to be processed in step S1302serves as the area of attention, and the object in which the mask areasare displayed superimposed on the touch UI 204 is determined. Forexample, as the object following the object 1101 in FIG. 11B, the screendisplayed if the object 1102 is selected as the object that is thetarget of attention is shown in FIG. 12E. In the present embodiment, asshown in FIG. 12E, with respect to the objects 1103 and 1104 to be thetarget of attention after the object 1102 that is the target ofattention, the masks 1106 and 1107 that are each opaque are displayedsuperimposed on the page 400. Additionally, because the object 1101 isdisplayed as the object that has already been the target of attentionbefore the display of the object 1102, a semi-transparent mask 1109 thatis a semi-transparent white is displayed superimposed on the page 400.With this manner, the object that has already been displayed isdisplayed in an inconspicuous state while indicating that the display isdone, and the object 1102 that is currently the target of attention canbe displayed in an easily viewable state. Note that the display statesof the objects in FIG. 12E, and FIG. 11D and FIG. 12F described beloware illustrated as FIG. 15B. In FIG. 12E, because the object 1102(object ID: 02) is a new target of attention, the attention state flagand the already displayed flag in FIG. 15B are set to 1. Additionally,the semi-transparent mask 1109 in FIG. 12E is displayed if the attentionstate flag is set to 0 and the already displayed flag is set to 1 inFIG. 15B. Similarly, the opaque mask in FIG. 12E is displayed if theattention state flag is set to 0 and the already displayed flag is setto 0 in FIGS. 15A to 15C.

Next, in step S1305, the annotation change processing is performed.Simply explained, deletion of the annotation is performed. For example,the annotation 1108 entered in FIG. 11C is deleted in the annotationchange processing, and it is displayed as shown in FIG. 12E afterupdating the display state in step S1306. With this manner, if theobject 1102 to be displayed next is the target of attention, theannotations are not displayed superimposed. Therefore, a display that iseasily viewable for the viewer is made possible. Additionally, asanother annotation change processing, the transmittance that is theattribute information of the annotation may be changed, not deleting theannotation. For example, the transmittance of the attribute informationof the annotation 1108 entered in FIG. 11C changes to 95% as shown inFIG. 16B. Accordingly, after updating the display state in step S1306,the annotation 1108 is displayed in a transmitted state as shown in FIG.12F, so that the display that is easily viewable for the viewer is madepossible without interfering with the display of the object 1102.

Note that, for comparison, a display example in the case where theannotation change processing in step S1305 was not performed isillustrated in FIG. 11D. As compared with the case where the annotationchange processing was performed, this case shows that the object 1102becomes difficult to be seen due to the annotation 1108. Note that, inthe present embodiment, although the transmittance is changed to 95%,for example, it may be possible to make the object completely invisibleby setting the transmittance to 100%, and the transmittance can be setto any value. In the case of changing the transmittance, the annotationis not deleted, so that the annotation can be left with the applicationimage data. Therefore, the corresponding object ID is held as theattribute information, the annotation can be displayed in conjunctionwith the object of attention if the application image data will bedisplayed later, or if the previous button selection processingdescribed below is performed. Additionally, in the present embodiment,although the transmittance of the annotation is changed, a display thatis easily viewable may be performed by changing the line width, thecolor, or a layer, and the annotation change processing is not limitedto the change of the transmittance.

Next, in step S1306, the display state of the touch UI 204 is updatedbased on the processing of the determined attention area and theannotation change processing. In contrast, in step S1302, a case wherethe next object could not been read (if the next selectable object isabsent) indicates a state in which a final object in the page hasalready been read. Accordingly, in step S1303, it is determined that thedisplay processing for all the objects has been finished, and theprocess proceeds to step S1307. Subsequently, in step S1307, the displayorder control unit 305 finishes the presentation display mode and hidesthe masks 1105 to 1109, and switches the mode to the page display mode.

Next, in step S1308, because the display mode is the page display mode,the syntax of the application image data stored in the RAM 211 isanalyzed, and the next page and the object included therein are read.Next, in step S1309, the display order control unit 305 determineswhether or not the next page could be read in step S1308. That is, itdetermines whether or not the page has been finished depending on theresult indicating whether the next page could be read or not. If thenext page could be read (NO), the process proceeds to step S1310. Incontrast, if the final page of the application image data stored in theRAM 211 has been read and the next readable page is absent in S1308 (thenext page cannot be read) (YES), the process proceeds to step S1311.Subsequently, in step S1311, the syntax of the application image datastored in the RAM 211 is analyzed, and the head page and the objectincluded therein are read. Next, in step S1310, the display change eventprocessing unit 304 controls the display magnification of the page 400in accordance with the height of the touch UI 204, controls the startpoint of the page 400, and determines the display range of the page.Subsequently, in step S1306, the display state of the touch UI 204 isupdated.

Next, a description will be given of “previous selection processing(previous button selection processing)” executed if “previous button”403 is tapped (specified) by the user, with reference to the flowchartillustrating the previous selection processing in FIG. 17. The previousselection processing performs the processes that are the same as stepS1301, step S1303, step S1306, step S1307, step S1309, and S1310 in FIG.17. Therefore, here, only steps S1601 to S1605, which are different fromthe process in FIG. 17, will be described.

First, in step S1301, if the current display mode is the presentationdisplay mode, the process proceeds to step S1601. Subsequently, in stepS1601, the display order control unit 305 selects the object to be nextdisplayed based on the sentence structure tree from among all theobjects in the page currently being read, and reads the object. In thepresent embodiment, the display order on the document tree structure ifthe previous button 403 is specified is the object on the lower layerthat is the end in the document tree structure, and then the object inthe lower layer that is a previous layer from the end of the object,that is, the order from the end of the lower layer. For example, in FIG.8, the object to be first read is T7, which is the lower layer object atthe end. If the “previous selection processing” is performed in a statewhere the object of T7 is being displayed on the touch UI 204 in thepresentation display mode, the object of T6 is read due to the presenceof T6 on the same layer. Additionally, if the “previous selectionprocessing” is performed in a state where the object of T6 is beingdisplayed, the object of T5 that is present at the end of the lowerlayer of the layer V1, which is present on the layer same as V2 that isthe upper layer of T6, is read because another object on the layer sameas T6 is absent. Note that also in the “previous selection processing”,it may be possible that only the object having a drawing element isselected in a state similar to the “next selection processing”, or onlyobjects having a specific attribute are selected, or objects excludingonly the specific attribute are selected. Additionally, it may bepossible to select only an object that has a drawing element on thelower layer and does not have the drawing element as a single element,for example, H1 or V1.

Next, if the next object could be read in step S1303 (if the selectableobject is present: NO), the process proceeds to step S1602.Subsequently, in the attention area determination processing in stepS1602, the previous object that was read in step S1601 serves as thearea of attention, and the object in which the mask areas are displayedsuperimposed on the touch UI 204 is determined. For example, as anobject that is previous to the object 1102 in FIG. 12E, the screen thatis displayed if the object 1101 is selected as the object of the targetof attention is shown in FIG. 12G. In the present embodiment, as shownin FIG. 12G, because the object 1102 has already been displayed as theobject of the target of attention, the opaque mask 1105 is replaced bythe semi-transparent mask that is semi-transparent white, and displayedsuperimposed on the page 400 in FIG. 11B. With this manner, the object1101 that is currently the target of attention can be displayed in aneasily viewable state while indicating the object that has already beendisplayed in the inconspicuous state. Note that the display state ofFIG. 12G is represented as FIG. 15C.

Next, in step S1603, the annotation change processing is performed.Simply explained, in the state same as in step S1305, the deletion ofannotation is performed. Note that, in the case of holding theannotation, not deleting the annotation, the transmittance that is theattribute information of the annotation may be varied. For example,regarding the annotation 1108 having a transmittance of 95%, which isassociated with the object 1101 in FIG. 12F, its transmittance ischanged to 0% as shown in FIG. 12H. With this manner, if the previousobject is selected by the previous selection processing, the associatedannotation can be displayed in an easily viewable state again.

If the next object could not been read in step S1303 (if the selectableobject is absent: YES), the process proceeds to step S1307, the mode isswitched to the page display mode, and the process proceeds to stepS1604. Subsequently, in step S1604, because the display mode is the pagedisplay mode, the syntax of the application image data stored in the RAM211 is analyzed, and the previous page and the object included in it isread. Next, if the page that can be next read is absent (the next pagecannot be read) in step S1309 (YES), the process proceeds to step S1605.Subsequently, in step S1605, the syntax of the application image datastored in the RAM 211 is analyzed, and the final page and the objectincluded in it are read.

As described above, according to the present embodiment, if the objectsare displayed in order, the annotation is displayed in conjunction withthe display of the objects, and therefore the display that is easilyviewable and suitable for the presentation while entering the annotationin the displayed document is allowed. Additionally, the enteredannotation is associated with the object and stored with the document,so that if the document is reproduced later, the viewer can browse thedocument while displaying the annotation in a state similar to asituation in which the presenter explained the document while enteringthe annotation. Consequently, the viewer can browse the document as ifthe presenter is explaining it while entering the annotation in thepresentation even if the viewer sees the document alone.

Second Embodiment

In the first embodiment, it was described in which the display of theannotation during the presentation display mode is changed inconjunction with the transition of the object of attention in theannotation display control unit 307. Consequently, only the annotationcorresponding to the object that is currently receiving attention isdisplayed with highlights, and the object and the annotation to whichthe viewer needs to pay attention could be displayed in the easilyviewable state. On the other hand, the presenter sometimes wants toperform the enlarged display of the page 400 by the pinch-out operation,without using the next button 402 and previous button 403. For example,the presenter can enlarge the screen as shown in FIG. 19B by thepinch-out operation. In this case, because the next button 402 andprevious button 403 were not used, the attention area control unit 306was not able to identify the attention area.

In contrast, in the second embodiment, a process in which the area ofattention is defined and the display control of the annotation isperformed in the case where the presenter enlarges the page 400 by thepinch-out operation will be described. Hereinafter, a description willbe given by focusing on differences with the first embodiment withreference to FIG. 18 and FIGS. 19A and 19B. FIG. 18 is a flowchartillustrating the annotation change processing during the enlargeddisplay. FIGS. 19A and 19B are diagrams illustrating an example of thescreen of the annotation in the case of the enlarged display.

First, in step S1701, the gesture event processing unit 302 determineswhether or not the page 400 is enlarged by a pinch-out event. If thepage has been enlarged by the pinch-out event (YES), the processproceeds to step S1702, and if the page has not been enlarged (NO), theprocess ends. Next, in step S1702, the attention area control unit 306performs attention area determination processing that determines theobject that is contained within the screen during the enlarged displayof the page 400. For example, in FIG. 19A, the entire page 400 isdisplayed, and the dashed lines 1101 to 1104 show the objects. However,the dashed lines are displayed for clearly showing the object, and theyare not displayed on the actual page 400. Additionally, the enteredannotations are annotations 1108 and 1801, wherein the annotation 1108is associated with the object 1101, and the annotation 1801 isassociated with the object 1103. At this time, the page 400 that hasbeen enlarged by the user is shown in FIG. 19B and the object area thatis displayed during the enlargement are determined to be the area ofattention because of the display area of the page 400 and the displayarea of the object during the enlargement display. Specifically, theobjects 1102 to 1104 shown by the broken lines in FIG. 19B serve as theareas of attention during the enlarged display.

Next, in step S1703, the annotation display control unit 307 changes thedisplay of the annotation so as to display only the annotationassociated with the area that was determined to be the area of attentionin step S1702. For example, in FIG. 19B, because the objects 1102 to1104 become the areas of attention, only the annotation 1801 associatedwith the object 1103 is displayed, the transmittance of the annotation1108 is set to 100%, and it is displayed so that the viewer cannot seeit. Subsequently, in step S1704, based on the processing of thedetermined attention area and the change processing of the annotation,the display state of the touch UI 204 is updated.

As described above, according to the present embodiment, in addition tothe configuration of the first embodiment, the attention area during theenlargement/reduction of the page is specified, only the annotationcorresponding to the object of the (displayed) area of attention isdisplayed, and the display state is changed so as not to display theannotation that does not correspond. Therefore, the presenter caneffectively show the document without distracting viewers.

Third Embodiment

In the first embodiment and the second embodiment, the description wasgiven of the present embodiments in which the display of the annotationis changed in conjunction with the area of the object to be displayedwith attention (area of attention). Thereby, only the annotationcorresponding to the object that is currently receiving the attention isdisplayed with highlights, so that the object and annotation to whichattention is paid can be displayed to the user in the easily viewablestate.

In contrast, in the third embodiment, in an application in which theobject is automatically animation-displayed in the order specified bythe user in advance, a description will be given of processing ofperforming the display control of the annotation in conjunction with thestate transition of the animation display. Here, the animation displayindicates that the objects are displayed in accordance with the orderand the operation that are specified by the user in advance.Hereinafter, with reference to FIGS. 20 to 22, a description will begiven focusing on differences with the first embodiment.

FIG. 20 is a flow chart that associates the annotations with the statetransition indicating the display order of the animation. FIGS. 21A and21B illustrate the display order in a case where the object and theannotation shown in FIGS. 22A and 22B are animation-displayed in order.Because the operation of the animation is not directly related to thepresent embodiment, the detailed description will be omitted. FIGS. 22Aand 22B are diagrams illustrating a display example of the object andthe annotation. Note that the order display of the animation can bechanged by the next button 402 and the previous button 403.Specifically, if the object 1102 having display order 02 (object ID: 02)in FIG. 22A is displayed, the next button 402 is tapped, and theanimation-display of the object 1103 having display order 03 (object ID:03) is performed.

Here, with reference to FIG. 20, first, the annotation image isgenerated in step S1901. Since the generation of the annotation imagehas same processes as steps S1203 shown in FIG. 13, and the detaileddescription will be omitted. In order to simplify the description, acase is assumed in which the drawing button 404 is tapped by the user ifthe object 1103 having display order 03 (object ID: 03) in FIG. 21A isbeing displayed, and the annotation is entered.

Next, in step S1902, the display order of the animation is changed.Specifically, as shown in FIG. 21B, the object 1104 having display order04 (object ID: 04) in FIG. 21A is changed to the animation display ofthe annotation 1801 (annotation ID: 01). Subsequently, the display order04 that was originally present in FIG. 21A is changed to display order05. Consequently, the animation display of the annotation 1801(annotation ID: 01) can be performed between the animation display ofthe object 1103 (object ID: 03) and the object 1104 (object ID: 04).

Specifically, the animation display of the annotation is added to theoriginal animation so that the reproduction in a state similar to thecase if the presentation was performed while entering the annotation,even if reproducing the application later is allowed. Note that theanimation display operation of the annotation is performed such that theannotation image is displayed by pop-up, but the present invention isnot limited to this, and other operations may be used if the operationenables newly displaying the annotation. Additionally, the annotation1801 having display order 04 (annotation ID: 01) in FIG. 21B isdisplayed by changing its transmittance if the display is switched tothe animation of the following order.

As described above, according to the present embodiment, in addition tothe configurations of the first embodiment and the second embodiment,the display of the annotation is added to the display order of theanimation. Consequently, the user can display the annotation imageentered by the presenter in conjunction with the original animation.

OTHER EMBODIMENTS

Embodiment (s) of the present invention can also be realized by acomputer of a system or apparatus that reads out and executes computerexecutable instructions (e.g., one or more programs) recorded on astorage medium (which may also be referred to more fully as a‘non-transitory computer-readable storage medium’) to perform thefunctions of one or more of the above-described embodiment (s) and/orthat includes one or more circuits (e.g., application specificintegrated circuit (ASIC)) for performing the functions of one or moreof the above-described embodiment (s), and by a method performed by thecomputer of the system or apparatus by, for example, reading out andexecuting the computer executable instructions from the storage mediumto perform the functions of one or more of the above-describedembodiment(s) and/or controlling the one or more circuits to perform thefunctions of one or more of the above-described embodiment (s). Thecomputer may comprise one or more processors (e.g., central processingunit (CPU), micro processing unit (MPU)) and may include a network ofseparate computers or separate processors to read out and execute thecomputer executable instructions. The computer executable instructionsmay be provided to the computer, for example, from a network or thestorage medium. The storage medium may include, for example, one or moreof a hard disk, a random-access memory (RAM), a read only memory (ROM),a storage of distributed computing systems, an optical disk (such as acompact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD)™),a flash memory device, a memory card, and the like.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures and functions

This application claims the benefit of Japanese Patent Application No.2014-161697, filed Aug. 7, 2014, which is hereby incorporated byreference wherein in its entirety.

What is claimed is:
 1. An image display apparatus that displays imagedata including a plurality of objects, the apparatus comprising: adisplay unit configured to focus-display an object from among theplurality of objects; an input unit configured to input annotation tothe object; and a change unit configured to change a display of theannotation depending on the focus-displayed object if an objectdifferent from the object corresponding to the annotation that has beeninput by the input unit is focus-displayed.
 2. The image displayapparatus according to claim 1, wherein if a user selects an object fromamong the plurality of objects and the display unit focus-displays theobject, the change unit changes the display so as to display only theannotation corresponding to the selected object.
 3. The image displayapparatus according to claim 1, further comprising: a specifying unitconfigured to specify an order for displaying the object from among theplurality of objects, wherein, in a case where the display unitfocus-displays each of the plurality of objects in order in a forwarddirection or a backward direction in accordance with the specifiedorder, the change unit changes the display of the annotationcorresponding to the object being displayed if the next object that isdifferent from the object being displayed is focus-displayed.
 4. Theimage display apparatus according to claim 3, wherein the specifyingunit further specifies the order including the input annotation, andwherein if the display unit automatically performs the focus-display inorder in accordance with the specified order, the change unit changesthe display of the annotation depending on the order including theannotation.
 5. The image display apparatus according to claim 3, whereinthe specifying unit specifies the order depending on a position of eachof the objects in the image data.
 6. The image display apparatusaccording to claim 1, wherein the change unit changes the display of theannotation by lowering the transmittance of the annotation correspondingto the object being displayed and increasing the transmittance of theannotation corresponding to the object different from the object beingdisplayed.
 7. A method for controlling an image display apparatus thatdisplays image data including a plurality of objects, the methodcomprising: focus-displaying an object from among the plurality ofobjects; inputting annotation to the object; and changing a display ofthe annotation depending on the focus-displayed object if an objectdifferent from the object corresponding to the annotation that has beeninput in the inputting is focus-displayed.
 8. A non-transitory storagemedium storing a readable program for causing a computer to execute acontrol method of an image display apparatus that displays image dataincluding a plurality of objects, the method comprising:focus-displaying an object from among the plurality of objects;inputting annotation to the object; and changing a display of theannotation depending on the focus-displayed object if an objectdifferent from the object corresponding to the annotation that has beeninput in the inputting is focus-displayed.